Transient Atomic Markov Chains with a Denumerable Number of States

On Weak Lumpability of Denumerable Markov Chains

We consider weak lumpability of denumerable discrete or continuous time Markov chains. Firstly, we are concerned with irreducible recurrent positive and R-positive Markov chains evolving in discrete time. We study the properties of the set of all initial distributions of the starting chain leading to an aggregated homogeneous Markov chain with respect to a partition of the state space. In parti...

Potentials for Denumerable Markov Chains

Probabilistic generalizations of classical potential theory have been worked out by J. L. Doob [ l ] and G. A. Hunt [2]. Specialized to discrete processes, this provides a potential theory for transient Markov chains, analogous to the theory of the Newtonian potential. We provide a unified potential theory for all denumerable Markov chains; as applied to recurrent chains, the theory generalizes...

A new machine replacement policy based on number of defective items and Markov chains

  A novel optimal single machine replacement policy using a single as well as a two-stage decision making process is proposed based on the quality of items produced. In a stage of this policy, if the number of defective items in a sample of produced items is more than an upper threshold, the machine is replaced. However, the machine is not replaced if the number of defective items is less than ...

Transient Markov Chains with Stationary Measures

A note on exponential stability of the nonlinear filter for denumerable Markov chains

We study asymptotic stability of the optimal filter with respect to its initial conditions. We show that exponential stability of the nonlinear filter holds for a large class of denumerable Markov chains, including all finite Markov chains, under the assumption that the observation function is one-to-one and the observation noise is sufficiently small. Ergodicity of the signal process is not as...